{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "view-in-github"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/NeuromatchAcademy/course-content/blob/master/tutorials/W2D2_LinearSystems/student/W2D2_Tutorial1.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "# Neuromatch Academy 2020, Week 2, Day 2, Tutorial 1\n",
    "\n",
    "# Linear dynamical systems\n",
    "\n",
    "**Content Creators**: Bing Wen Brunton, Alice Schwarze\n",
    "\n",
    "**Content Reviewers**: Norma Kuhn, Karolina Stosio, John Butler, Matthew Krause, Ella Batty, Richard Gao, Michael Waskom"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "---\n",
    "# Tutorial Objectives\n",
    "\n",
    "In this tutorial, we will be learning about behavior of dynamical systems -- systems that evolve in time -- where the rules by which they evolve in time are described precisely by a differential equation.\n",
    "\n",
    "Differential equations are equations that express the **rate of change** of the state variable $x$. One typically describes this rate of change using the derivative of $x$ with respect to time ($dx/dt$) on the left hand side of the differential equation:\n",
    "\n",
    "$$\\frac{dx}{dt} = f(x)$$\n",
    "\n",
    "A common notational short-hand is to write $\\dot{x}$ for $\\frac{dx}{dt}$. The dot means \"the derivative with respect to time\".\n",
    "\n",
    "Today, the focus will be on **linear dynamics**, where $f(x)$ is a linear function of $x$. In Tutorial 1, we will:\n",
    "\n",
    "* Explore and understand the behavior of such systems where $x$ is a single variable\n",
    "* Consider cases where $\\mathbf{x}$ is a state vector representing two variables. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "---\n",
    "# Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code"
   },
   "outputs": [],
   "source": [
    "# Imports\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from scipy.integrate import solve_ivp  # numerical integration solver"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "colab": {},
    "colab_type": "code"
   },
   "outputs": [],
   "source": [
    "#@title Figure settings\n",
    "import ipywidgets as widgets       # interactive display\n",
    "%config InlineBackend.figure_format = 'retina'\n",
    "plt.style.use(\"https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/nma.mplstyle\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "colab": {},
    "colab_type": "code"
   },
   "outputs": [],
   "source": [
    "#@title Helper functions\n",
    "def plot_trajectory(system, params, initial_condition, dt=0.1, T=6,\n",
    "                    figtitle=None):\n",
    "\n",
    "    \"\"\"\n",
    "    Shows the solution of a linear system with two variables in 3 plots.\n",
    "    The first plot shows x1 over time. The second plot shows x2 over time.\n",
    "    The third plot shows x1 and x2 in a phase portrait.\n",
    "\n",
    "    Args:\n",
    "      system (function): a function f(x) that computes a derivative from\n",
    "                         inputs (t, [x1, x2], *params)\n",
    "      params (list or tuple): list of parameters for function \"system\"\n",
    "      initial_condition (list or array): initial condition x0\n",
    "      dt (float): time step of simulation\n",
    "      T (float): end time of simulation\n",
    "      figtitlte (string): title for the figure\n",
    "\n",
    "    Returns:\n",
    "      nothing, but it shows a figure\n",
    "    \"\"\"\n",
    "\n",
    "    # time points for which we want to evaluate solutions\n",
    "    t = np.arange(0, T, dt)\n",
    "\n",
    "    # Integrate\n",
    "    # use built-in ode solver\n",
    "    solution = solve_ivp(system,\n",
    "                     t_span=(0, T),\n",
    "                     y0=initial_condition, t_eval=t,\n",
    "                     args=(params),\n",
    "                     dense_output=True)\n",
    "    x = solution.y\n",
    "\n",
    "    # make a color map to visualize time\n",
    "    timecolors = np.array([(1 , 0 , 0, i)  for i in t / t[-1]])\n",
    "\n",
    "    # make a large figure\n",
    "    fig, (ah1, ah2, ah3) = plt.subplots(1, 3)\n",
    "    fig.set_size_inches(10, 3)\n",
    "\n",
    "    # plot x1 as a function of time\n",
    "    ah1.scatter(t, x[0,], color=timecolors)\n",
    "    ah1.set_xlabel('time')\n",
    "    ah1.set_ylabel('x1', labelpad=-5)\n",
    "\n",
    "    # plot x2 as a function of time\n",
    "    ah2.scatter(t, x[1], color=timecolors)\n",
    "    ah2.set_xlabel('time')\n",
    "    ah2.set_ylabel('x2', labelpad=-5)\n",
    "\n",
    "    # plot x1 and x2 in a phase portrait\n",
    "    ah3.scatter(x[0,], x[1,], color=timecolors)\n",
    "    ah3.set_xlabel('x1')\n",
    "    ah3.set_ylabel('x2', labelpad=-5)\n",
    "    #include initial condition is a blue cross\n",
    "    ah3.plot(x[0,0], x[1,0], 'bx')\n",
    "\n",
    "    # adjust spacing between subplots\n",
    "    plt.subplots_adjust(wspace=0.5)\n",
    "\n",
    "    # add figure title\n",
    "    if figtitle is not None:\n",
    "      fig.suptitle(figtitle, size=16)\n",
    "\n",
    "\n",
    "def plot_streamplot(A, ax, figtitle=None):\n",
    "    \"\"\"\n",
    "    Show a stream plot for a linear ordinary differential equation with\n",
    "    state vector x=[x1,x2] in axis ax.\n",
    "\n",
    "    Args:\n",
    "      A (numpy array): 2x2 matrix specifying the dynamical system\n",
    "      figtitle (string): title for the figure\n",
    "\n",
    "    Returns:\n",
    "      nothing, but shows a figure\n",
    "    \"\"\"\n",
    "\n",
    "    # sample 20 x 20 grid uniformly to get x1 and x2\n",
    "    grid = np.arange(-20, 21, 1)\n",
    "    x1, x2 = np.meshgrid(grid, grid)\n",
    "\n",
    "    # calculate x1dot and x2dot at each grid point\n",
    "    x1dot = A[0,0] * x1 + A[0,1] * x2\n",
    "    x2dot = A[1,0] * x1 + A[1,1] * x2\n",
    "\n",
    "    # make a colormap\n",
    "    magnitude = np.sqrt(x1dot ** 2 + x2dot ** 2)\n",
    "    color = 2 * np.log1p(magnitude) #Avoid taking log of zero\n",
    "\n",
    "    # plot\n",
    "    plt.sca(ax)\n",
    "    plt.streamplot(x1, x2, x1dot, x2dot, color=color,\n",
    "                   linewidth=1, cmap=plt.cm.cividis,\n",
    "                   density=2, arrowstyle='->', arrowsize=1.5)\n",
    "    plt.xlabel(r'$x1$')\n",
    "    plt.ylabel(r'$x2$')\n",
    "\n",
    "    # figure title\n",
    "    if figtitle is not None:\n",
    "        plt.title(figtitle, size=16)\n",
    "\n",
    "    # include eigenvectors\n",
    "    if True:\n",
    "        # get eigenvalues and eigenvectors of A\n",
    "        lam, v = np.linalg.eig(A)\n",
    "\n",
    "        # get eigenvectors of A\n",
    "        eigenvector1 = v[:,0].real\n",
    "        eigenvector2 = v[:,1].real\n",
    "\n",
    "        # plot eigenvectors\n",
    "        plt.arrow(0, 0, 20*eigenvector1[0], 20*eigenvector1[1],\n",
    "                  width=0.5, color='r', head_width=2,\n",
    "                  length_includes_head=True)\n",
    "        plt.arrow(0, 0, 20*eigenvector2[0], 20*eigenvector2[1],\n",
    "                  width=0.5, color='b', head_width=2,\n",
    "                  length_includes_head=True)\n",
    "\n",
    "def plot_specific_example_stream_plots(A_options):\n",
    "    \"\"\"\n",
    "    Show a stream plot for each A in A_options\n",
    "\n",
    "    Args:\n",
    "      A (list): a list of numpy arrays (each element is A)\n",
    "\n",
    "    Returns:\n",
    "      nothing, but shows a figure\n",
    "    \"\"\"\n",
    "    # get stream plots for the four different systems\n",
    "    plt.figure(figsize=(10,10))\n",
    "\n",
    "    for i, A in enumerate(A_options):\n",
    "\n",
    "        ax = plt.subplot(2, 2, 1+i)\n",
    "        # get eigenvalues and eigenvectors\n",
    "        lam, v = np.linalg.eig(A)\n",
    "\n",
    "        # plot eigenvalues as title\n",
    "        # (two spaces looks better than one)\n",
    "        eigstr = \",  \".join([f\"{x:.2f}\" for x in lam])\n",
    "        figtitle =f\"A with eigenvalues\\n\"+ '[' + eigstr + ']'\n",
    "        plot_streamplot(A, ax, figtitle=figtitle)\n",
    "\n",
    "        # Remove y_labels on righthand plots\n",
    "        if i%2:\n",
    "          ax.set_ylabel(None)\n",
    "        if i<2:\n",
    "          ax.set_xlabel(None)\n",
    "\n",
    "        plt.subplots_adjust(wspace=0.3, hspace=0.3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "---\n",
    "# Section 1: One-dimensional Differential Equations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 518
    },
    "colab_type": "code",
    "outputId": "4aba3b3e-9d7b-46c0-9d50-88ffacecb95e"
   },
   "outputs": [],
   "source": [
    "#@title Video 1: Linear Dynamical Systems\n",
    "# Insert the ID of the corresponding youtube video\n",
    "from IPython.display import YouTubeVideo\n",
    "video = YouTubeVideo(id=\"87z6OR7-DBI\", width=854, height=480, fs=1)\n",
    "print(\"Video available at https://youtu.be/\" + video.id)\n",
    "video"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "\n",
    "\n",
    "Let's start by reminding ourselves of a one-dimensional differential equation in $x$ of the form \n",
    "\n",
    "$$\\dot{x} = a x$$ \n",
    "\n",
    "where $a$ is a scalar. \n",
    "\n",
    "Solutions for how $x$ evolves in time when its dynamics are governed by such a differential equation take the form\n",
    "\n",
    "$$x(t) = x_0\\exp(a t)$$\n",
    "\n",
    "where $x_0$ is the **initial condition** of the equation -- that is, the value of $x$ at time $0$.\n",
    "\n",
    "To gain further intuition, let's explore the behavior of such systems with a simple simulation. We can simulate an ordinary differential equation by approximating or modelling time as a discrete list of time steps $t_0, t_1, t_2, \\dots$, such that $t_{i+1}=t_i+dt$. We can get the small change $dx$ over a small duration $dt$ of time from the definition of the differential:\n",
    "\n",
    "$$ \\\n",
    "\\begin{eqnarray}\n",
    "  \\dot x &=& \\frac{dx}{dt} \\\\\n",
    "  dx     &=& \\dot x\\, dt  \\\\\n",
    "\\end{eqnarray}\n",
    "$$\n",
    "\n",
    "So, at each time step $t_i$, we compute a value of $x$, $x(t_i)$, as the sum of the value of $x$ at the previous time step, $x(t_{i-1})$ and a small change $dx=\\dot x\\,dt$:\n",
    "\n",
    "$$x(t_i)=x(t_{i-1})+\\dot x(t_{i-1}) dt$$\n",
    "\n",
    "This very simple integration scheme, known as **forward Euler integration**, works well if $dt$ is small and the ordinary differential equation is simple. It can run into issues when the ordinary differential equation is very noisy or when the dynamics include sudden big changes of $x$. Such big jumps can occur, for example, in models of excitable neurons. In such cases, one needs to choose an integration scheme carefully. However, for our simple system, the simple integration scheme should work just fine! \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "## Exercise 1 (1A): Forward Euler Integration\n",
    "\n",
    "In this exercise, we will complete a function, ``integrate_exponential``, to compute the solution of the differential equation $\\dot{x} = a x$ using forward Euler integration. We will then plot this solution over time.\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "code",
    "colab": {},
    "colab_type": "code"
   },
   "outputs": [],
   "source": [
    "def integrate_exponential(a, x0, dt, T):\n",
    "  \"\"\"Compute solution of the differential equation xdot=a*x with\n",
    "  initial condition x0 for a duration T. Use time step dt for numerical\n",
    "  solution.\n",
    "\n",
    "  Args:\n",
    "    a (scalar): parameter of xdot (xdot=a*x)\n",
    "    x0 (scalar): initial condition (x at time 0)\n",
    "    dt (scalar): timestep of the simulation\n",
    "    T (scalar): total duration of the simulation\n",
    "\n",
    "  Returns:\n",
    "    ndarray, ndarray: `x` for all simulation steps and the time `t` at each step\n",
    "  \"\"\"\n",
    "\n",
    "  # Initialize variables\n",
    "  t = np.arange(0, T, dt)\n",
    "  x = np.zeros_like(t, dtype=complex)\n",
    "  x[0] = x0 # This is x at time t_0\n",
    "\n",
    "  # Step through system and integrate in time\n",
    "  for k in range(1, len(t)):\n",
    "\n",
    "    ###################################################################\n",
    "    ## Fill out the following then remove\n",
    "    raise NotImplementedError(\"Student exercise: need to implement simulation\")\n",
    "    ###################################################################\n",
    "\n",
    "    # for each point in time, compute xdot from x[k-1]\n",
    "    xdot = ...\n",
    "\n",
    "    # Update x based on x[k-1] and xdot\n",
    "    x[k] = ...\n",
    "\n",
    "  return x, t\n",
    "\n",
    "\n",
    "# choose parameters\n",
    "a = -0.5    # parameter in f(x)\n",
    "T = 10      # total Time duration\n",
    "dt = 0.001  # timestep of our simulation\n",
    "x0 = 1.     # initial condition of x at time 0\n",
    "\n",
    "# Uncomment once you've filled in code above\n",
    "# x, t = integrate_exponential(a, x0, dt, T)\n",
    "# plt.plot(t, x.real)\n",
    "# plt.xlabel('Time (s)')\n",
    "# plt.ylabel('x')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 430
    },
    "colab_type": "text",
    "outputId": "60ea62cd-f21f-4e7a-dd07-999bb274f5b4"
   },
   "source": [
    "[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W2D2_LinearSystems/solutions/W2D2_Tutorial1_Solution_af289c13.py)\n",
    "\n",
    "*Example output:*\n",
    "\n",
    "<img alt='Solution hint' align='left' width=558 height=414 src=https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/tutorials/W2D2_LinearSystems/static/W2D2_Tutorial1_Solution_af289c13_0.png>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "## Interactive Demo: Forward Euler Integration\n",
    "\n",
    " What happens when you change $a$? Try values where $a<0$ and $a>0$. \n",
    " \n",
    " The $dt$ is the step size of the forward Euler integration. Try $a = -1.5$ and increase $dt$. What happens to the numerical solution when you increase $dt$?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 495,
     "referenced_widgets": [
      "1a0cf406072246a3af2251e2f996b532",
      "9f7afb32f53943aa86188672440be02e",
      "7e7f2cf147204be691a1838f2b1895c0",
      "d52ec816cfbd4a218629cfada84d1883",
      "9e83b2122ffd4e719897ab4c83075f8e",
      "51f6674134234307b20573801d647285",
      "6e513e25df584a959b5485bb783046de",
      "16d47099343440199bbedd4c489155b0",
      "23bdd1e1526e4fef9ddf170f908ee4db",
      "386024b932a94d64a4fa7d4077bfb328"
     ]
    },
    "colab_type": "code",
    "outputId": "9d1523e7-2934-4d9a-bb61-aaa3fb3b19a6"
   },
   "outputs": [],
   "source": [
    "#@title\n",
    "\n",
    "#@markdown Make sure you execute this cell to enable the widget!\n",
    "\n",
    "T = 10      # total Time duration\n",
    "x0 = 1.     # initial condition of x at time 0\n",
    "\n",
    "@widgets.interact\n",
    "def plot_euler_integration(a=(-2.5, 1.5, .25), dt = widgets.SelectionSlider(options=[(\"%g\"%i,i) for i in np.arange(0.001, 1.001, 0.01)])):\n",
    "  # Have to do this clunky word around to show small values in slider accurately\n",
    "  # (from https://github.com/jupyter-widgets/ipywidgets/issues/259)\n",
    "\n",
    "  x, t = integrate_exponential(a, x0, dt, T)\n",
    "  plt.plot(t, x.real) # integrate_exponential returns complex\n",
    "  plt.xlabel('Time (s)')\n",
    "  plt.ylabel('x')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {},
    "colab_type": "text"
   },
   "source": [
    "[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W2D2_LinearSystems/solutions/W2D2_Tutorial1_Solution_4436181a.py)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "---\n",
    "# Section 2: Oscillatory Dynamics\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 518
    },
    "colab_type": "code",
    "outputId": "c963748e-6583-4f7f-9fa9-e44cb3e32dbe"
   },
   "outputs": [],
   "source": [
    "#@title Video 2: Oscillatory Solutions\n",
    "# Insert the ID of the corresponding youtube video\n",
    "from IPython.display import YouTubeVideo\n",
    "video = YouTubeVideo(id=\"vPYQPI4nKT8\", width=854, height=480, fs=1)\n",
    "print(\"Video available at https://youtu.be/\" + video.id)\n",
    "video"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "We will now explore what happens when $a$ is a complex number and has a non-zero imaginary component."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "## Interactive Demo: Oscillatory Dynamics\n",
    "In the following demo, you can change the real part and imaginary part of $a$ (so a = real + imaginary i)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 494,
     "referenced_widgets": [
      "c9675c18337a41da9e68cd5bf9683d56",
      "22c1fd48c70649f691a77843fccca625",
      "22f86a6ffd274dc8bfd35e969cfe7dc2",
      "389341d6e1d340f09c6aaa6726fe63de",
      "bbdaa5fcb7764091821b3ccd8a92f58b",
      "c408e29e49144a2a810fc1ed444b1eed",
      "4587e3d306af433eabb650ed64f93124",
      "3dfa00fa2d8f45b3ae556dcaeefe9d8b",
      "9fe1b8e83ac24c32a05d8e1a4e1b6fb9",
      "cb991e952dcd464fa156cd1bcd84a72e"
     ]
    },
    "colab_type": "code",
    "outputId": "3bb26499-7a74-4113-f47b-c2329deedd67"
   },
   "outputs": [],
   "source": [
    "#@title\n",
    "\n",
    "#@markdown Make sure you execute this cell to enable the widget!\n",
    "\n",
    "# parameters\n",
    "T = 5         # total Time duration\n",
    "dt = 0.0001      # timestep of our simulation\n",
    "x0 = 1.        # initial condition of x at time 0\n",
    "\n",
    "@widgets.interact\n",
    "def plot_euler_integration(real=(-2, 2, .2), imaginary=(-4, 7, .1)):\n",
    "\n",
    "  a = complex(real, imaginary)\n",
    "  x, t = integrate_exponential(a, x0, dt, T)\n",
    "  plt.plot(t, x.real) #integrate exponential returns complex\n",
    "  plt.grid(True)\n",
    "  plt.xlabel('Time (s)')\n",
    "  plt.ylabel('x')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "## Exercise 2 (1B): Finding Oscillatory Solutions\n",
    "Using the demo above, answer the following: \n",
    "- What values of $a$ produce dynamics that both ***oscillate*** and ***grow***?\n",
    "- What value of $a$ is needed to produce a stable oscillation of 0.5 Hertz (cycles/time units)?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {},
    "colab_type": "text"
   },
   "source": [
    "[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W2D2_LinearSystems/solutions/W2D2_Tutorial1_Solution_f72b0cca.py)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "---\n",
    "# Section 3: Deterministic Linear Dynamics in Two Dimensions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 518
    },
    "colab_type": "code",
    "outputId": "b67debec-1f35-4e96-c45f-708aaf2c980e"
   },
   "outputs": [],
   "source": [
    "#@title Video 3: Multi-Dimensional Dynamics\n",
    "# Insert the ID of the corresponding youtube video\n",
    "from IPython.display import YouTubeVideo\n",
    "video = YouTubeVideo(id=\"c_GdNS3YH_M\", width=854, height=480, fs=1)\n",
    "print(\"Video available at https://youtu.be/\" + video.id)\n",
    "video"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "Adding one additional variable (or _dimension_) adds more variety of behaviors. Additional variables are useful in modeling the dynamics of more complex systems with richer behaviors, such as systems of multiple neurons. We can write such a system using two linear ordinary differential equations:\n",
    "$$\n",
    "\\begin{eqnarray}\n",
    "  \\dot{x}_1 &=& {a}_{11} x_1 \\\\\n",
    "  \\dot{x}_2 &=& {a}_{22} x_2 \\\\\n",
    "\\end{eqnarray}\n",
    "$$\n",
    "So far, this system consists of two variables (e.g. neurons) in isolation. To make things interesting, we can add interaction terms:\n",
    "$$\n",
    "\\begin{eqnarray}\n",
    "  \\dot{x}_1 &=& {a}_{11} x_1 + {a}_{12} x_2 \\\\\n",
    "  \\dot{x}_2 &=& {a}_{21} x_1 + {a}_{22} x_2 \\\\\n",
    "\\end{eqnarray}\n",
    "$$\n",
    "We can write the two equations that describe our system as one (vector-valued) linear ordinary differential equation:\n",
    "\n",
    "$$\\dot{\\mathbf{x}} = \\mathbf{A} \\mathbf{x}$$\n",
    "\n",
    "For two-dimensional systems, $\\mathbf{x}$ is a vector with 2 elements ($x_1$ and $x_2$) and $\\mathbf{A}$ is a $2 \\times 2$ matrix with $\\mathbf{A}=\\bigg[\\begin{array} & a_{11} & a_{12} \\\\ a_{21} & a_{22} \\end{array} \\bigg]$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "## Exercise 3 (1C): Sample trajectories in 2 dimensions\n",
    "\n",
    "\n",
    "We want to simulate some **trajectories** of a given system and plot how  𝑥1  and  𝑥2  evolve in time. We will begin with this example system:\n",
    "\n",
    "$$\\dot{\\mathbf{x}} = \\bigg[\\begin{array} &2 & -5 \\\\ 1 & -2 \\end{array} \\bigg] \\mathbf{x}$$\n",
    "\n",
    "We will use an integrator from scipy, so we won't have to solve the system ourselves. We have a helper function, ``plot_trajectory``, that plots these trajectories given a system function. In this exercise, we will write the system function for a linear system with two variables. \n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code"
   },
   "outputs": [],
   "source": [
    "def system(t, x, a00, a01, a10, a11):\n",
    "    '''\n",
    "    Compute the derivative of the state x at time t for a linear\n",
    "    differential equation with A matrix [[a00, a01], [a10, a11]].\n",
    "\n",
    "    Args:\n",
    "      t (float): time\n",
    "      x (ndarray): state variable\n",
    "      a00, a01, a10, a11 (float): parameters of the system\n",
    "\n",
    "    Returns:\n",
    "      ndarray: derivative xdot of state variable x at time t\n",
    "    '''\n",
    "    #################################################\n",
    "    ## TODO for students: Compute xdot1 and xdot2 ##\n",
    "    ## Fill out the following then remove\n",
    "    raise NotImplementedError(\"Student exercise: say what they should have done\")\n",
    "    #################################################\n",
    "\n",
    "    # compute x1dot and x2dot\n",
    "    x1dot = ...\n",
    "    x2dot = ...\n",
    "\n",
    "    return np.array([x1dot, x2dot])\n",
    "\n",
    "\n",
    "T = 6 # total time duration\n",
    "dt = 0.1 # timestep of our simulation\n",
    "A = np.array([[2, -5],\n",
    "              [1, -2]])\n",
    "x0 = [-0.1, 0.2]\n",
    "\n",
    "# Uncomment once the system function is complete\n",
    "# plot_trajectory(system, [A[0,0],A[0,1],A[1,0],A[1,1]], x0, dt=dt, T=T)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 215
    },
    "colab_type": "text",
    "outputId": "80ef3c6e-af39-4ffc-c655-9f8fe6a4a91f"
   },
   "source": [
    "[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W2D2_LinearSystems/solutions/W2D2_Tutorial1_Solution_39b84465.py)\n",
    "\n",
    "*Example output:*\n",
    "\n",
    "<img alt='Solution hint' align='left' width=690 height=198 src=https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/tutorials/W2D2_LinearSystems/static/W2D2_Tutorial1_Solution_39b84465_0.png>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "## Interactive Demo: Varying A\n",
    "\n",
    "We will now use the function we created in the last exercise to plot trajectories with different options for A. What kinds of qualitatively different dynamics do you observe? Hint: Keep an eye on the x-axis and y-axis!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 247,
     "referenced_widgets": [
      "9ccc54af937c4bea810b19a14c423106",
      "6a4e8a8565d2456c8137a9e90acf603a",
      "e4393c7ecca24ed0b0785ad0c93fdd69",
      "719b8cad0bfc4e8c80b36715ab8da850",
      "6b0b019d12f54dcf8b579161e974d55f",
      "36ec7594def748928849ade9f59b7775",
      "6d8cb31b1d134d25aaa0844c921f183e"
     ]
    },
    "colab_type": "code",
    "outputId": "a469bdca-e910-4a61-f410-e23f159b849e"
   },
   "outputs": [],
   "source": [
    "#@title\n",
    "\n",
    "#@markdown Make sure you execute this cell to enable the widget!\n",
    "\n",
    "# parameters\n",
    "T = 6      # total Time duration\n",
    "dt = 0.1   # timestep of our simulation\n",
    "x0 = np.asarray([-0.1, 0.2])        # initial condition of x at time 0\n",
    "\n",
    "A_option_1 = [[2, -5],[1, -2]]\n",
    "A_option_2 = [[3,4], [1, 2]]\n",
    "A_option_3 = [[-1, -1], [0, -0.25]]\n",
    "A_option_4 = [[3, -2],[2, -2]]\n",
    "\n",
    "@widgets.interact\n",
    "def plot_euler_integration(A = widgets.Dropdown(\n",
    "  options=[A_option_1, A_option_2, A_option_3, A_option_4, None],\n",
    "  value=A_option_1\n",
    ")):\n",
    "  if A:\n",
    "    plot_trajectory(system, [A[0][0],A[0][1],A[1][0],A[1][1]], x0, dt=dt, T=T)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {},
    "colab_type": "text"
   },
   "source": [
    "[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W2D2_LinearSystems/solutions/W2D2_Tutorial1_Solution_80f8ef7f.py)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "## Interactive Demo: Varying Initial Conditions\n",
    "\n",
    "We will now vary the initial conditions for a given $\\mathbf{A}$:\n",
    "\n",
    "$$\\dot{\\mathbf{x}} = \\bigg[\\begin{array} &2 & -5 \\\\ 1 & -2 \\end{array} \\bigg] \\mathbf{x}$$\n",
    "\n",
    "What kinds of qualitatively different dynamics do you observe? Hint: Keep an eye on the x-axis and y-axis!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 247,
     "referenced_widgets": [
      "af82b8666b874f3384b6ef50ad54154c",
      "8ae3d5bc2e244cc2b4f2b4cdbad05a98",
      "8d19e3b185d14f769d1009c93e7002a3",
      "fbe234eb5e464320a0d72266443079ff",
      "708f514338154ed5aeebff6cdf248a3f",
      "92fa78b707394cbba31b5ef9016bcfc0",
      "aa0a005282e44f58bc22a8385ad942cb"
     ]
    },
    "colab_type": "code",
    "outputId": "a6e4402b-a552-4532-c837-c8f989f1292a"
   },
   "outputs": [],
   "source": [
    "#@title\n",
    "\n",
    "#@markdown Make sure you execute this cell to enable the widget!\n",
    "\n",
    "# parameters\n",
    "T = 6      # total Time duration\n",
    "dt = 0.1   # timestep of our simulation\n",
    "x0 = np.asarray([-0.1, 0.2])        # initial condition of x at time 0\n",
    "A = [[2, -5],[1, -2]]\n",
    "\n",
    "x0_option_1 = [-.1, 0.2]\n",
    "x0_option_2 = [10, 10]\n",
    "x0_option_3 = [-4, 3]\n",
    "\n",
    "@widgets.interact\n",
    "def plot_euler_integration(x0 = widgets.Dropdown(\n",
    "  options=[x0_option_1, x0_option_2, x0_option_3, None],\n",
    "  value=x0_option_1\n",
    ")):\n",
    "  if x0:\n",
    "    plot_trajectory(system, [A[0][0],A[0][1],A[1][0],A[1][1]], x0, dt=dt, T=T)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {},
    "colab_type": "text"
   },
   "source": [
    "[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W2D2_LinearSystems/solutions/W2D2_Tutorial1_Solution_5f343601.py)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "---\n",
    "# Section 4: Stream Plots"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "It's a bit tedious to plot trajectories one initial condition at a time!\n",
    "\n",
    "Fortunately, to get an overview of how a grid of initial conditions affect trajectories of a system, we can use a _stream plot_. \n",
    "\n",
    "We can think of a initial condition ${\\bf x}_0=(x_{1_0},x_{2_0})$  as coordinates for a position in a space. For a 2x2 matrix $\\bf A$, a stream plot computes at each position $\\bf x$ a small arrow that indicates $\\bf Ax$ and then connects the small arrows to form _stream lines_. Remember from the beginning of this tutorial that $\\dot {\\bf x} = \\bf Ax$ is the rate of change of $\\bf x$. So the stream lines indicate how a system changes. If you are interested in a particular initial condition ${\\bf x}_0$, just find the correponding position in the stream plot. The stream line that goes through that point in the stream plot indicates ${\\bf x}(t)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "Using some helper functions, we show the stream plots for each option of A that you examined in the earlier interactive demo. We included the eigenvectors of $\\bf A$ as a red line (1st eigenvalue) and a blue line (2nd eigenvalue) in the stream plots.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 718
    },
    "colab_type": "code",
    "outputId": "bcf55384-12b1-459d-de4d-24d1199c71bc"
   },
   "outputs": [],
   "source": [
    "A_option_1 = np.array([[2, -5], [1, -2]])\n",
    "A_option_2 = np.array([[3,4], [1, 2]])\n",
    "A_option_3 = np.array([[-1, -1], [0, -0.25]])\n",
    "A_option_4 = np.array([[3, -2], [2, -2]])\n",
    "\n",
    "A_options = [A_option_1, A_option_2, A_option_3, A_option_4]\n",
    "plot_specific_example_stream_plots(A_options)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "### Think\n",
    "\n",
    "What is special about the direction in which the principal eigenvector points? And how does the stability of the system relate to the corresponding eigenvalues? (Hint: Remember from your [introduction to linear algebra](https://www.youtube.com/watch?v=PFDu9oVAE-g&list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab&index=15&t=0s) that, for matrices with real eigenvalues, the eigenvectors indicate the lines on which $\\bf Ax$ is parallel to $\\bf x$ and real eigenvalues indicate the factor by which $\\bf Ax$ is streched or shrunk compared to $\\bf x$.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code"
   },
   "outputs": [],
   "source": [
    "#### Jot downs some thoughts here ###"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {},
    "colab_type": "text"
   },
   "source": [
    "[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W2D2_LinearSystems/solutions/W2D2_Tutorial1_Solution_4b6aef45.py)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "---\n",
    "# Summary\n",
    "\n",
    "In this tutorial, we learned:\n",
    "\n",
    "* How to simulate the trajectory of a dynamical system specified by a differential equation $\\dot{x} = f(x)$ using a forward Euler integration scheme. \n",
    "* The behavior of a one-dimensional linear dynamical system $\\dot{x} = a x$ is determined by $a$, which may be a complex valued number. Knowing $a$, we know about the stability and oscillatory dynamics of the system.\n",
    "* The dynamics of high-dimensional linear dynamical systems $\\dot{\\mathbf{x}} = \\mathbf{A} \\mathbf{x}$ can be understood using the same intuitions, where we can summarize the behavior of the trajectories using the eigenvalues and eigenvectors of $\\mathbf{A}$."
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "include_colab_link": true,
   "name": "W2D2_Tutorial1",
   "provenance": [],
   "toc_visible": true
  },
  "kernel": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
